VC dimension and inner product space induced by Bayesian networks

VC Dimension Bounds for Product Unit Networks

A product unit is a formal neuron that multiplies its input values instead of summing them. Furthermore, it has weights acting as exponents instead of being factors. We investigate the complexity of learning for networks containing product units. We establish bounds on the Vapnik-Chervonenkis (VC) dimension that can be used to assess the generalization capabilities of these networks. In particu...

Inner Product Spaces for Bayesian Networks

Bayesian networks have become one of the major models used for statistical inference. We study the question whether the decisions computed by a Bayesian network can be represented within a low-dimensional inner product space. We focus on two-label classification tasks over the Boolean domain. As main results we establish upper and lower bounds on the dimension of the inner product space for Bay...

VC Dimension of Neural Networks

This paper presents a brief introduction to Vapnik-Chervonenkis (VC) dimension, a quantity which characterizes the difficulty of distribution-independent learning. The paper establishes various elementary results, and discusses how to estimate the VC dimension in several examples of interest in neural network theory.

VC dimension of ReLU networks

Neural Networks with Quadratic VC Dimension

This paper shows that neural networkswhich use continuousactivation functions have VC dimension at least as large as the square of the number of weights w. This results settles a long-standing open question, namely whether the well-known O(w log w) bound, known for hardthreshold nets, also held for more general sigmoidal nets. Implications for the number of samples needed for valid generalizati...